FIT to Monitor Feed Quality
نویسندگان
چکیده
While there has been significant focus on collecting and managing data feeds, it is only now that attention is turning to their quality. In this paper, we propose a principled approach to online data quality monitoring in a dynamic feed environment. Our goal is to alert quickly when feed behavior deviates from expectations. We make contributions in two distinct directions. First, we propose novel enhancements to the DFMS architecture to permit a publish-subscribe approach to incorporate data quality modules into the DFMS architecture. Second, we propose novel temporal extensions to standard statistical techniques to adapt them to online feed monitoring for outlier detection and alert generation at multiple scales along three dimensions: aggregation at multiple time intervals to detect at varying levels of sensitivity; multiple lengths of data history for varying the speed at which models adapt to change; and multiple levels of monitoring delay to address lagged data arrival. FIT, or Feed Inspection Tool, is the result of a successful implementation of our approach. We present several case studies outlining the effective deployment of FIT in real applications along with user testimonials.
منابع مشابه
A Generalized Linear Statistical Model Approach to Monitor Profiles
Statistical process control methods for monitoring processes with univariate ormultivariate measurements are used widely when the quality variables fit to known probabilitydistributions. Some processes, however, are better characterized by a profile or a function of qualityvariables. For each profile, it is assumed that a collection of data on the response variable along withthe values of the c...
متن کاملDesign of a Fuzzy Feed-forward Controller for Monitor HAGC System of Cold Rolling Mill
In this study we propose a novel monitor hydraulic automatic gauge control (HAGC) system based on fuzzy feedforward controller. This is used in the development of cold rolling mill automation system to improve the quality of cold strip. According to features/ properties of entry steel strip like its average yield stress, width of strip, and desired exit thickness, this controller realizes the c...
متن کاملIt’s All About Political Incentives: Democracy and the Renewable Feed-In Tariff
Demand for renewable energy is booming. Scholars often attribute this success to feed-in tariffs (FITs), which mandate that energy utilities pay a premium to renewable electricity producers and guarantee grid access for them. Why have so many countries, including least developed ones, adopted these policies? We hypothesize that democratic governments have political incentives to adopt the FIT b...
متن کاملAnalytical Performance and Quality Control of a Glucose Monitor System
Background and Objective: The reliability and validity of monitors for self-monitoring of blood glucose are debated. We evaluated the analytical performance of Accu-check Active (Boehringer Mannheim, Roche) which is one of the most commonly used monitors in Iran. Material and Methods: We compared the monitor readings with the reference values by percentage of values within...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- PVLDB
دوره 8 شماره
صفحات -
تاریخ انتشار 2015